A fast randomized algorithm for the approximation of matrices

We introduce a randomized procedure that, given an m×n matrix A and a positive integer k, approximates A with a matrix Z of rank k. The algorithm relies on applying a structured l×m random matrix R to each column of A, where l is an integer near to, but greater than, k. The structure of R allows us to apply it to an arbitrary m× 1 vector at a cost proportional to m log(l); the resulting procedure can construct a rank-k approximation Z from the entries of A at a cost proportional to mn log(k)+ l2(m+ n). We prove several bounds on the accuracy of the algorithm; one such bound guarantees that the spectral norm ‖A− Z‖ of the discrepancy between A and Z is of the same order as √ max{m,n} times the (k+ 1)st greatest singular value σk+1 of A, with small probability of large deviations. In contrast, the classical pivoted “QR” decomposition algorithms (such as Gram–Schmidt or Householder) require at least kmn floating-point operations in order to compute a similarly accurate rank-k approximation. In practice, the algorithm of this paper runs faster than the classical algorithms, even when k is quite small or large. Furthermore, the algorithm operates reliably independently of the structure of the matrix A, can access each column of A independently and at most twice, and parallelizes naturally. Thus, the algorithm provides an efficient, reliable means for computing several of the greatest singular values and corresponding singular vectors of A. The results are illustrated via several numerical examples. © 2007 Elsevier Inc. All rights reserved.